test: assert Level-1 candidate membership and account for every rejection - #728

This should be some CMS/CS sketches with heap?

I ran the pinned ASAPPlanner/backend compiler and exported every compilable candidate. For topk by (label_0) (3, data), it emits an exact fallback and a backend-local CurrentSeries(data, by label_0) -> TopK(k=3) readout. The test now selects and asserts the local plan. CMS/CS heaps estimate frequency-heavy hitters, while this PromQL TopK ranks the current sample values and must retain the original series labels; using a frequency sketch directly would change the query. A certified candidate-membership sketch plus exact reranking could be a separate optimization.

Confirmed, and the enumerator already does this — the test was the thing that was silent.

Planner exposes a CountSketchWithHeap candidate for spatial-topk over the same current-series snapshot. It is enumerated, then rejected before pricing with selected summary readout has no certified accuracy guarantee, because this fixture supplies no distinct-item bound and no score separation. The exact Sort → Limit program is what binds.

family: None used to mean the test asserted nothing here. It now declares the shape explicitly:

"spatial-topk" => &[(
    "CountSketchWithHeap",
    Resolution::MustReject(PolicyReason::NoCertifiedGuarantee),
)],

So the heap candidate is required to be present and required to be refused for that specific reason — if Planner stops exposing it, or refuses it for a different reason, the test fails. admission/spatial-topk.admission.json records it.

A certified companion fixture is not derivable from this generator; see the reply on the quantile thread and the note in required_summary_shapes.

what's the difference between rate and increase? Should they be two nodes?

They are separate physical roles here: the maintained Increase producer stores reset-aware counter state; the query DAG has an ExactReadout::Rate over that producer, applying the rate/window readout. The test now asserts that two-node producer/readout relationship rather than treating Increase as a Rate answer.

ExactKind has them as distinct variants (ASAPPlanner crates/types/src/post_asap/sketch.rs):

pub enum ExactKind { Sum, Count, Min, Max, Increase, Rate, IRate }

Increase is the counter-reset-aware delta; Rate is that delta divided by the window duration. So they are separate operators, not one node with a flag.

They are not two nodes in a plan, though. Every rate candidate in this fixture materializes as a single node — {"family":{"ExactAggregate":["Rate","Rate"]},"kind":"summary_agg","reduction":"PerEntity"} with a Readout{statistic:"Rate", logical_lookback_ms:"60000"} at the physical layer. The reset-aware delta is internal to the Rate accumulator rather than a separate Increase node feeding it.

Worth flagging: Increase appears in zero candidates across all ten queries, and so do Count, Min, Max and IRate. The ten-query workload only exercises Sum, Rate and quantiles. That is a real coverage gap in a branch named promql-exact-function-coverage — I think it belongs in a follow-up rather than here, but it should be a deliberate decision.

this should be, the inner node is rate, outer node is grouped sum exact aggregation.

Agreed. The actual selected chain is Logical Aggregate(sum by label_0) -> ExactReadout(rate) -> ReadMaterialization(Increase). The test now asserts the inner rate readout, the outer exact grouped Sum, and their edge.

Both placements are enumerated and both bind — the inventory is not committed to one.

• grouped-rate-4: query-time grouped Sum over complete per-series Rate readouts. SummaryBuild(Sum) appears only in the query DAG.
• grouped-rate-3: fixed-window precompute over complete per-series counter states. SummaryBuild{ExactAggregate:["Sum","Sum"]} runs at maintenance time; the query DAG is just Readout{statistic:"Sum"}.

In both, each series is finalized with PromQL Rate semantics before Sum groups the values — assert_native_grouped_rate checks that the Sum consumes a Readout{statistic:"Rate", logical_lookback_ms:"60000"}, so summing raw counters before per-series rates would fail.

The test now requires both placements to appear (grouped_rate_placements must contain {false, true}), rather than asserting one expected shape. Which of the two wins is #742;'s call, not this layer's.

One defect is visible here: a third precompute-Sum variant fails with Planner logical fragment does not match any original query subtree, and a fourth with a schema incompatibility. main is green on that path, so both are regressions from inside this stack. They are pinned in KNOWN_BINDING_DEFECTS with exact counts so they cannot be fixed or worsened silently.

Correction to my earlier reply on this thread.

I said the two bind_failed candidates here were regressions introduced inside the #737 → #728 stack, because main is green on these paths. That was wrong, and I should have diffed before asserting it.

residual_nodes is substantively identical on main. The only things this stack changed in that function are how the accuracy target is derived (previously hardcoded Exact) and the wording of the error message. main is green because its tests never plan these queries, not because the code is correct. The defect is older and wider than I claimed.

Root cause, now diagnosed rather than guessed. residual_nodes locates the query-time fragment by re-parsing every subtree of the original and comparing typed IR with ==. For this query the two sides differ in exactly one field:

residual   leaf scan columns: [ts, value, label_0]
candidate  leaf scan columns: [ts, value]

Everything else is identical: reduction, measures, output_names, range, source, predicates, time_index. Planner documents a PromQL leaf schema as usage-derived, "the (ts, value) floor + the labels the query references", and marks it closed: false because it deliberately does not enumerate the row. rate(data[1m]) carries label_0 only because the enclosing sum by (label_0) references it; re-parsing that subtree on its own cannot know that.

Fixed on main in #781 by comparing open leaf schemas by containment (a prefix, so positional ColumnIds keep their meaning) while everything else stays exact. The pre-existing false-match guards, which discriminate on matchers, range and metric name, all pass unchanged.

KNOWN_BINDING_DEFECTS and the review README have been corrected to say this.

this should be fused by ASAPPlanner into one sum, over time and spatial dimensions.

The actual Planner candidate is currently Logical Aggregate(sum by label_0) -> ExactReadout(sum) -> per-entity Sum producer; it has not fused time and spatial aggregation into one grouped temporal Sum producer. I changed the expected contract to the fused producer, so the level-1 test now fails and names this gap. This comment remains an implementation blocker rather than being marked resolved.

The fused candidate exists and is the one that binds cheapest — grouped-temporal-sum-0 is a single operator over both dimensions:

{"kind":"summary_agg","family":{"ExactAggregate":["Sum","Sum"]},"reduction":{"Reduce":[2]}}

with "window_size":60 (the 1m temporal range) and "grouping_labels":{"labels":["label_0"]} (the spatial grouping) on that same node, "partitioning":"grouped". No separate per-entity stage.

The unfused shape is also enumerated (grouped-temporal-sum-3): summary_agg{Sum, reduction:"PerEntity"} → FinalizeExactAccumulator → a distinct Aggregate{sum, Reduce} node. Both are legal frontiers, so Level 1 keeps both and asserts neither as the winner.

On your earlier point about the 1-minute read: the grouped branch now checks readout_lookback_ms = 60_000 and that window_ms matches the materialization's stored_window_ms(), so a 5-second pane cannot pass as the 1-minute answer.

Also here: one candidate fails with Planner logical fragment does not match any original query subtree, recorded in KNOWN_BINDING_DEFECTS.

There should be inner node of rate, outter node for some sketches for topk

The actual local chain is TopKSelection(k=3, by label_0) -> ExactReadout(rate) -> ReadMaterialization(Increase). It computes rate first and then selects TopK locally. A CMS/CS frequency heap is not itself an exact TopK of rate values; a sketch candidate stage would require a completeness certificate and exact rerank. The test asserts the current semantically correct local chain, and all candidate plans are exported for review.

That shape is enumerated — four candidates of it, in fact:

inner Readout{statistic:"Rate"} → outer KeyedSummaryBuild{Sketch:[CmsWithHeap | CountSketchWithHeap]} → KeyedReadout → Sort → Limit.

Two are query-time, two are fixed-window candidates that finalize Rate and build the heap during precompute. All four are rejected before pricing with no certified accuracy guarantee, so exact Sort → Limit binds.

The test now requires both families and that specific resolution:

"topk-rate" => &[
    ("CmsWithHeap", Resolution::MustReject(PolicyReason::NoCertifiedGuarantee)),
    ("CountSketchWithHeap", Resolution::MustReject(PolicyReason::NoCertifiedGuarantee)),
],

plus the pre-existing structural checks that the maintenance DAG contains Rate + KeyedSummaryBuild while the query DAG contains KeyedReadout and not KeyedSummaryBuild, and that ranking sorts on the finalized rate value rather than a timestamp or label.

this should be DDSketch / DDSketch

Agreed on the intended composition: q=0.9 and q=0.5 quantile-sketch readouts over the same per-series window, followed by local division. I added that target DAG contract. The current Planner emits only ExactFallback candidates, so the level-1 test remains red for this query; I am not treating fallback as a correct answer.

That is what happens, and the test now says so instead of asserting nothing.

quantile-ratio-0 is the bound candidate: one summary_agg{Sketch:[DDSketch,...]} feeding two summary_estimate{Quantile} nodes (q=0.9 and q=0.5) into a binary/Div. So DDSketch / DDSketch, sharing a single producer.

KLL is attempted and discarded before admission — its composed guarantee does not satisfy EpsilonDelta{epsilon:0.01, delta:0.01} for either quantile. The other two candidates degrade to exact_fallback as a result. Exact (non-fallback) never appears.

The existing assertions already check the operand structure, the shared per-entity materialization at readout_lookback_ms = 60_000, and that both readouts come from a quantile-sketch family. What changed is that a KLL rejection for any reason other than an accuracy-target miss now fails the test rather than passing quietly.